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Abstract. In this paper, we derive an explicit formula for constructing the 
confidence interval of binomial parameter with guaranteed coverage probabil- 
ity. The formula overcomes the limitation of normal approximation which 
is asymptotic in nature and thus inevitably introduce unknown errors in ap- 
plications. Moreover, the formula is very tight in comparison with classic 
Clopper-Pearson's approach from the perspective of interval width. Based 
on the rigorous formula, we also obtain approximate formulas with excellent 
performance of coverage probability. 



1. Classic Confidence Intervals 

The construction of confidence interval of binomial parameter is frequently en- 
countered in communications and many other areas of science and engineering. 
Clopper and Pearson [3] has provided a rigorous approach for constructing confi- 
dence interval. However, the computational complexity involved with this approach 
is very high. The standard technique is to use normal approximation which is not 
accurate for rare events, especially in the context of studying the bit error rate of 
communication systems, blocking probability of communication networks and prob- 
ability of instability of uncertain dynamic systems. Moreover, it has been recently 
proven by Brown, Cai and DasGupta [TJI2 that the standard normal approximation 
approach is persistently poor. The coverage probability of the confidence interval 
can be significantly below the specified confidence level even for very large sample 
sizes. Since in many situations, it is desirable to quickly construct a confidence 
interval with guaranteed coverage probability, our goal is to derive a simple and 
rigorous formula for confidence interval construction. 

Let the probability space be denoted as (Q, F, P) where fi, F, P are the sample 
space, the algebra of events and the probability measure respectively. Let X be 
a Bernoulli random variable with distribution Pr{X = 1} = Fx, Pr{X — 0} = 
1— Px where ¥ x G (0, 1). Let the sample size N and confidence parameter S C (0, 1) 
be fixed. We refer an observation with value 1 as a successful trial. Let K denote 
the number of successful trials during the N i.i.d. sampling experiments. Let 
k = K(u>) where w is a sample point in the sample space f2. 
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1.1. Clopper- Pearson Confidence Limits. The classic Clopper-Pearson lower 
confidence limit LN,k,8 and upper confidence limit Un.Ic,s are given respectively by 

def f if k = . TT def / 1 if k = N 

Ln ^ s = \ p if k > and U »> k > B - { p if k < N 

where p £ (0, 1) is the solution of the following equation 

(i.i) iif!)^ 1 -^" 1 ^ 

3=0 ^ 3 ' 

and p £ (0, 1) is the solution of the following equation 

(i-2) eCH^ 1 

3=0 V J J 

The probabilistic implication of the confidence limits can be illustrated as follows: 
Define random variable L : O, — ► [0, 1] by L(u>) — L N K ^ g Vcj £ Q and random 
variable U : Q — ► [0, 1] by [/(w) = U NtK ^ >s € fi. Then 

Pr{L < P x < U} > 1 - 5. 

The exact value of Pr{L < Px < Uy is referred as the coverage probability. Ac- 
cordingly, we refer Pr{Px < L or Px > as the error probability. 

1.2. Normal Approximation. It is easy to sec that the equations (|1.1[) and (|1.2p 

are very hard to solve and thus the confidence limits are very difficult to determine 
using Clopper-Pearson's approach. For large sample size, it is computationally 
prohibitive. To get around the difficulty, normal approximation has been widely 
used to develop simple approximate formulas (see, for example, [TJ [2j [6] and the 
references therein ). The basis of the normal approximation is the Central Limit 
Theorem, i.e., 

{I— - P I 1 
1 N X] < z \ = 2$(z) - 1 
/ Px(l-Px) J 

where z > and $(.) is the normal distribution function. Let Z§_ be the critical 

2 

value such that &(Zs) = 1 — It follows that 

lim P r /£-Z. A /^M<P JC <^ + Z.^EM\ = i-f, 
w^oo |JV A N 5 v A J 

i.e., 



I 2 Kj 2 7 1 / JV W ' , 2 



hmx^oo Pr { ^7 — < Px < 

s. 



I 



Since w for sufficiently large sample size A, the lower and upper confidence 
limits can be estimated respectively as 

N ~- V A 



BINOMIAL CONFIDENCE INTERVAL 



3 



and 

tt ~ k _i_ 7 a / Iv (-*- ~ at) 
U ~N +Z t)l N ■ 

The critical problem with the normal approximation is that it is of asymptotic 
nature. It is not clear how large the sample size is sufficient for the approximation 
error to be negligible. Such an asymptotic approach is not good enough for many 
practical applications involving rare events. 

2. Rigorous Formula 

It is desirable to have a simple formula which is rigorous and very tight for the 
confidence interval construction. We now propose the following simple formula for 
constructing the confidence limits. 

Theorem 1. Define 



and 



(2.2) W(A:)1S f - + - ^ Y ^ ^ jfc = 0,l,---,JV 

with 9 = tt^t. Then Pr{£(K) <P X < U(K)\ >l-5. Moreover, 

Bin s 

£(fc) < Liv.fe.a < U Ntk ,s < U(k). 

Remark 1. £(fc) and W(fc) are tight bounds for the classic Clopper-Pearson confi- 
dence limits Ljv,fc,<5 and UN.k,s (See Figures 1-12). A bisection search can be per- 
formed based on such bounds for computing the classic Clopper-Pearson confidence 
limits. 

To show Theorem [TJ we need some preliminary results. The following Lemma Q] 
is due to Massart [7]. 

Lemma 1. Pr{f >P x +e} < cxp (- 2(Px+f j v ( f_ Px _ f ) ) for allee (0,1-P X ). 

Of course, the above upper bound holds trivially for e > 1 — Fx- Thus, LemmafT] 
is actually true for any e > 0. 

K A ^ I \ 



Lemma 2. Pr { § < F x - e} < exp (- 2(Px _|j V (1 _ Px + | ) j for all e > 0. 

Proof. Define Y = 1 — X. Then Py = 1 — F X - At the same time when we are 
conducting N i.i.d. experiments for X, we are also conducting N i.i.d. experiments 
for Y. Let the number of successful trials of the experiments for Y be denoted as 
Ky. Obviously Ky = N — K. Applying Lemma Q] to Y, we have 



Pr <^ — - > Py + e } < exp 



N J V 2(Py + f) (1 



2 



It follows that 



(N-K ) ( Ne 2 

Pr <^ > 1 - P x + e > < exp 



N ~ A J" r V 2(1-P X + |) [l-(l-Pjc) 
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The proof is thus completed by observing that Pr { > 1 - P x + ( } = Pr { § < Px 
□ 

The following lemma can be found in [4]. 

Lemma 3. £j=o (T) 3 '' 3 ^ — X ) N ~'' decreases monotonically with respect to x € 
(0,1) for k = 0,1,- ■■ ,N. 

Lemma 4. £* =Q (*V(1 - < ex P (ix+ ^^ x _^ ) Vx e ( * , 1) 

for k = 0,1,- ■■ ,N. 

Proof. Consider binomial random variable X with parameter Px > 77- Let K 
be the number of successful trials during N i.i.d. sampling experiments. Then 

E ( N ) P x(l - Vx) N ~ j = ^{K < k}. 

Note that Pr{K < k} = Pr {f < P x - (Px - 77)}- Applying Lemma with 
e = Px - 77 > 0, we have 



j.o V 2(P x -^p-)(l-P x 

_fc 



N(P X ~ -|) 2 



exp V 2(|Px + ^)(i-|Px-^r);" 

Since the argument holds for arbitrary binomial random variable X with Px > 77 , 
the proof of the lemma is thus completed. □ 

Lemma 5. £fco QV(l - > 1 - cxp V, e 

(0,£)/orfc = l,-- - ,JV. 

Proof. Consider binomial random variable X with parameter Px < 77. Let JsT 
be the number of successful trials during N i.i.d. sampling experiments. Then 

E (^) p * (1 - Px)A " J = Pr{JT < fc} = Pr {§ < p * + ( ^ - Vx) } ■ 

Applying Lemma [1] with e — — Px > 0, we have that 



,=o V 2(V X + ^ L ) (1-Px 

_fe N 

N ' 



- 1 exp J n r k 2 



2 (Fx + 3A7) (1 



Since the argument holds for arbitrary binomial random variable X with Px < 77 , 
the proof of the lemma is thus completed. □ 

Lemma 6. Let < k < N . Then Ljv k s < Un k s- 



BINOMIAL CONFIDENCE INTERVAL 



5 



Proof. Obviously, the lemma is true for k= 0, N. We consider the case that 
I < k < N — I. Let S{N,k,x) = £ fc =0 (^)x j (1- x) N ~^> for x £ (0,1). Notice that 



N 



S(N,k,p) = S(N,k-l,p) + I V IjJ^l -#*- A 



Thus 



S(N,k- l,p) -S(N,k- l,p) = 1 



5 /iV 
2 ~ V fc 



Notice that 8 £ (0, 1) and that p £ (0, 1), we have that 



S(N, k-l,p)- S(N, k - l,p) = 1-8+ (l^til - V) N - k > 0. 

By Lemma [3j S(N,k — l,x) decreases monotonically with respect to x, we have 
p <p and complete the proof of the lemma. □ 



We are now in the position to prove Theorem [T] It can be easily verified that 
U N .k,8 < U{k) for k = 0, N. We need to show that U N>k>s < U{k) for < k < N. 
Straightforward computation shows that U (k) is the only root of equation 



exp 



N{x-±Y \ 8 



2 d x + m) ( 1 ~ \ x ~ mU 2 



with respect to x £ (-^,oo). There are two cases: U{k) > 1 and U(k) < 1. If 
U(k) > 1 then Un,u,8 < U(k) is trivially true. We only need to consider the case 
that & < U{k) < 1. In this case, it follows from Lemma [4] that 

V ( N ) mk) y { i-u(k))^ < exp (--, f -) = *. 

few " V a(i«(*) + ^)(i-fw(*)-^V 2 

Recall that 



we have 



£ ; ) c W 1 - cr w) JV " i >£( ■ [Wd-w(fc)) 

Therefore, by Lemma|31 we have that UN,k.s < U(k) for < k < N. Thus, we have 
shown that UN,k,s < 9 for all k. 

Similarly, by Lemma [5] and Lemma O we can show that Ljv,fe,<5 — C(k). By 
Lemma El we have C(k) < LN,k,s < Unm,s < U{k). Finally, the proof of Theo- 
rem[T]is completed by invoking the probabilistic implication of the Clopper-Pearson 
confidence interval. 
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3. Numerical Experiments and Empirical Formulas 

In comparison with the Clopper-Pearson's approach, our approach is very tight 
from the perspective of interval width (see, for example, Figures 1-12). Moreover, 
there is no comparison on the computational complexity. Our formula is simple 
enough for hand calculation. 

Our numerical results are in agreement with the discovery made by Brown, 
Cai and DasGupta [HE]- It can be seen from Figures 21-27 that the coverage 
probability of confidence intervals obtained by the standard normal approximation 
can be substantially lower than the specified confidence level 1 — 5 (This is true 
even when the condition for applying the rule of thumb, i.e., NFx(l — Px) > 5, 
is satisfied). Moreover, the situation is worse for smaller confidence parameter 5. 
See, for example, Figures 25-27, if one wishes to make an inference with an error 
frequency less than one out of 1000, using the normal approximation can lead to a 
frequency of error higher than 100 out of 1000. In light of the excessively high error 
rate of inference caused by the normal approximation, the rigorous formula may be 
a better choice. The rigorous formula guarantees the error probability below the 
specify level S. It should be noted that the rigorous formula is conservative (with 
actual error probability around 10% to 20% of the requirement). 

It should be noted that by tuning the parameter 9 in the rigorous formula, one 
can obtained simple formulas which meet the specified confidence levels. For exam- 
ple, to construct confidence interval with confidence parameter 5 = 0.05, 0.01, 0.001, 
we can simply compute C(k) and I4(k) defined in Theorem [1] with 9 — ^, -|, g 
respectively (The values of 9 presented here are not optimal. Better coverage per- 
formance can be achieved by a fine tuning of 9) . More specifically, 

Pr{f + | ^-Vya-f) <¥x<§ + f izgl^EE Mj^o.gs; 

Pr{# + f i^mp3> < Px < k + 3 Mf±^W(El>} „ o.99; 

Prjf + | < Vx < § + 3 ^+vW(MF) } M . 999 . 

Confidence limits computed by these formulas for different N and S are depicted 
by Figures 13-20. It is interesting to note that, in most situations, the confidence 
limits computed by our empirical formulas almost coincide with the corresponding 
limits derived by Clopper-Pearson method. The numerical investigation of the 
coverage probability of different confidence intervals is shown in Figures 21-27. It 
can be seen that the empirical formulas have excellent coverage performance. 
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FIGURE 1. Confidence Interval (N = 10, 5 = 0.05.) 
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FIGURE 2. Confidence Interval (N = 10, S = 0.01.) 




FIGURE 3. Confidence Interval (N = 50, S = 0.05.) 
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FIGURE 4. Confidence Interval (N = 50, 5 = 0.01.) 




Figure 5. Confidence Interval (N = 100, S = 0.05.) 
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Figure 6. Confidence Interval (N = 100, 5 = 0.01.) 




Figure 8. Confidence Interval (N = 500, 5 = 0.01.) 
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Figure 9. Confidence Interval (N = 1000, S = 0.05.) 
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Figure 10. Confidence Interval (N = 1000, <5 = 0.01.) 
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Figure 11. Confidence Interval (N = 5000, <5 = 0.05.) 
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Figure 12. Confidence Interval (N = 5000, <5 = 0.01.) 
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Figure 13. Confidence Interval (N = 50, 6 = 0.05.) 
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Figure 14. Confidence Interval (N = 50, 6 = 0.01.) 
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Figure 15. Confidence Interval (N = 100, S = 0.05.) 




Figure 16. Confidence Interval (N = 100, 6 = 0.01.) 
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Figure 17. Confidence Interval (N = 500, S = 0.05.) 




Number of Successful Trials 



Figure 18. Confidence Interval (N = 500, 6 = 0.01.) 
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Figure 19. Confidence Interval (N = 1000, <5 = 0.05.) 
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Figure 20. Confidence Interval (N = 1000, <5 = 0.01.) 
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FIGURE 21. Error Probability (F x = 0.5, S = 0.05. A - Normal, 
B - Empirical, C - Rigorous ) 
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Figure 22. Error Probability (F x = 0.01, S = 0.05. A - Normal, 
B - Empirical, C - Rigorous) 
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FIGURE 24. Error Probability (P x = 0.01, <5 = 10~ 2 . A - Nor- 
mal, B - Empirical, C - Rigorous) 



BINOMIAL CONFIDENCE INTERVAL 



19 




100 200 300 400 500 600 700 800 900 1000 

Sample Size N 



Figure 25. Error Probability (P x = 0.5, S = 1CT 3 . A - Normal, 
B - Empirical, C - Rigorous) 
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FIGURE 26. Error Probability (F x = 10~ 2 , S = 1(T 3 . A - 
Normal, B - Empirical, C - Rigorous) 
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FIGURE 27. Error Probability (P x = 10 -5 , S = 10" 3 . A - 
Normal, B - Empirical, C - Rigorous) 
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